Generalized Entropy Concentration for Counts

We consider the phenomenon of entropy concentration under linear constraints in a discrete setting, using the “balls and bins” paradigm, but without the assumption that the number of balls allocated to the bins is known. Therefore instead of frequency vectors and ordinary entropy, we have count vectors with unknown sum, and a certain generalized entropy. We show that if the constraints bound the allowable sums, this suffices for concentration to occur even in this setting. The concentration can be either in terms of deviation from the maximum generalized entropy value, or in terms of the norm of the difference from the maximum generalized entropy vector. Without any asymptotic considerations, we quantify the concentration in terms of various parameters, notably a tolerance on the constraints which ensures that they are always satisfied by an integral vector. Generalized entropy maximization is not only compatible with ordinary MaxEnt, but can also be considered an extension of it, as it allows us to address problems that cannot be formulated as MaxEnt problems.